Blur aware metric depth estimation with multi-focus plenoptic cameras

While a traditional camera only captures one point of view of a scene, a plenoptic or light-field camera, is able to capture spatial and angular information in a single snapshot, enabling depth estimation from a single acquisition. In this paper, we present a new metric depth estimation algorithm using only raw images from a multi-focus plenoptic camera. The proposed approach is especially suited for the multi-focus configuration where several micro-lenses with different focal lengths are used. The main goal of our blur aware depth estimation (BLADE) approach is to improve disparity estimation for defocus stereo images by integrating both correspondence and defocus cues. We thus leverage blur information where it was previously considered a drawback. We explicitly derive an inverse projection model including the defocus blur providing depth estimates up to a scale factor. A method to calibrate the inverse model is then proposed. We thus take into account depth scaling to achieve precise and accurate metric depth estimates. Our results show that introducing defocus cues improves the depth estimation. We demonstrate the effectiveness of our framework and depth scaling calibration on relative depth estimation setups and on real-world 3D complex scenes with ground truth acquired with a 3D lidar scanner.Comment: 21 pages, 12 Figures, 3 Table

Leveraging blur information for plenoptic camera calibration

This paper presents a novel calibration algorithm for plenoptic cameras, especially the multi-focus configuration, where several types of micro-lenses are used, using raw images only. Current calibration methods rely on simplified projection models, use features from reconstructed images, or require separated calibrations for each type of micro-lens. In the multi-focus configuration, the same part of a scene will demonstrate different amounts of blur according to the micro-lens focal length. Usually, only micro-images with the smallest amount of blur are used. In order to exploit all available data, we propose to explicitly model the defocus blur in a new camera model with the help of our newly introduced Blur Aware Plenoptic (BAP) feature. First, it is used in a pre-calibration step that retrieves initial camera parameters, and second, to express a new cost function to be minimized in our single optimization process. Third, it is exploited to calibrate the relative blur between micro-images. It links the geometric blur, i.e., the blur circle, to the physical blur, i.e., the point spread function. Finally, we use the resulting blur profile to characterize the camera's depth of field. Quantitative evaluations in controlled environment on real-world data demonstrate the effectiveness of our calibrations.Comment: arXiv admin note: text overlap with arXiv:2004.0774

Exploitation de l'information de flou dans les caméras plénoptiques : Application à l'étalonnage et à l'estimation métrique de profondeur

This thesis investigates the use of a vision sensor called a plenoptic camera for computer vision in robotics applications. To achieve this goal we place ourselves upstream of applications, and focus on its modelization to enable robust depth estimation.Plenoptic or light-field cameras are passive imaging systems able to capture both spatial and angular information about a scene in a single exposure. These systems are usually built upon a micro-lenses array (MLA) placed between a main lens and a sensor. Their design enables depth estimation from a single acquisition.The key contributions of this work lie in answering the questions "How can we link world space information to the image space information?" and more importantly, "How can we link image space information to world space information?". We address the first problem through the prism of calibration, by proposing a new camera model and a methodology to retrieve the intrinsic parameters of this model. We leverage blur information where it was previously considered as a drawback by explicitly modeling the defocus blur. We address the second one as the problematic of depth estimation, by proposing a metric depth estimation framework working directly with raw plenoptic images. It takes into account both correspondence and defocus cues. Our model generalizes to various configurations, including the multi-focus plenoptic camera (both in Galilean and Keplerian configuration), as well as to the single-focus and unfocused plenoptic camera. Our method gives accurate and precise depth estimates (a median relative error ranging from 1.27% to 4.75% of the distance). It outperforms state-of-the-art methods.Having a new complete camera model and enabling robust metric depth estimation from raw images only, opens the door to many new applications. It is an additional step towards practical use of plenoptic cameras in computer vision applications.Cette thèse propose d'étudier l'utilisation d'un capteur de vision appelé caméra plénoptique pour de la vision par ordinateur dans des applications robotiques. Plus précisément, pour atteindre cet objectif, nous nous plaçons en amont du côté applicatif, et nous nous concentrons sur sa modélisation pour permettre une estimation de profondeur robuste.Les caméras plénoptiques ou à champ de lumière sont des systèmes d'imagerie passifs capables de capturer les informations spatiales et angulaires d'une scène en une seule exposition. Ces systèmes sont généralement constitués d'une matrice de micro-lentilles (MLA) placée entre un objectif principal et un capteur. Leur conception permet l'estimation de la profondeur à partir d'une seule acquisition. Les contributions clés de ce travail résident dans la réponse à la question "Comment peut-on relier l'information de l'espace monde à celle de l'espace image ?" et surtout, "Comment peut-on relier l'information de l'espace image à celle de l'espace monde ?". Nous abordons la première par le prisme de l'étalonnage, en proposant un nouveau modèle de caméra et une méthodologie pour récupérer les paramètres intrinsèques de ce modèle. Nous exploitons l'information sur le flou de défocalisation là où il était auparavant considéré comme un inconvénient, en le modélisant explicitement. Nous abordons la deuxième problématique comme celle de l'estimation de profondeur, en proposant une méthode métrique d'estimation de profondeur fonctionnant directement avec des images brutes plénoptiques. Elle prend en compte à la fois les indices de correspondance et de défocalisation. Notre modèle se généralise à diverses configurations, y compris la caméra plénoptique multi-focales (en configuration galiléenne et keplérienne), ainsi qu'à la caméra plénoptique monofocale et non focalisée. Avec notre méthode, nous obtenons des estimations de profondeur répétables et exactes (de l'ordre de 1.27% à 4.75% de la distance à l'objet). Elle surpasse les résultats de l'état-de-l'art.Le fait de disposer d'un nouveau modèle complet de caméra et de permettre une estimation métrique robuste de la profondeur à partir d'images brutes uniquement ouvre la voie à de nombreuses nouvelles applications. Il s'agit d'un pas supplémentaire vers l'utilisation concrète de caméras plénoptiques dans les applications de vision par ordinateur

Exploitation de l'information de flou dans les caméras plénoptiques : Application à l'étalonnage et à l'estimation métrique de profondeur

This thesis investigates the use of a vision sensor called a plenoptic camera for computer vision in robotics applications. To achieve this goal we place ourselves upstream of applications, and focus on its modelization to enable robust depth estimation.Plenoptic or light-field cameras are passive imaging systems able to capture both spatial and angular information about a scene in a single exposure. These systems are usually built upon a micro-lenses array (MLA) placed between a main lens and a sensor. Their design enables depth estimation from a single acquisition.The key contributions of this work lie in answering the questions "How can we link world space information to the image space information?" and more importantly, "How can we link image space information to world space information?". We address the first problem through the prism of calibration, by proposing a new camera model and a methodology to retrieve the intrinsic parameters of this model. We leverage blur information where it was previously considered as a drawback by explicitly modeling the defocus blur. We address the second one as the problematic of depth estimation, by proposing a metric depth estimation framework working directly with raw plenoptic images. It takes into account both correspondence and defocus cues. Our model generalizes to various configurations, including the multi-focus plenoptic camera (both in Galilean and Keplerian configuration), as well as to the single-focus and unfocused plenoptic camera. Our method gives accurate and precise depth estimates (a median relative error ranging from 1.27% to 4.75% of the distance). It outperforms state-of-the-art methods.Having a new complete camera model and enabling robust metric depth estimation from raw images only, opens the door to many new applications. It is an additional step towards practical use of plenoptic cameras in computer vision applications.Cette thèse propose d'étudier l'utilisation d'un capteur de vision appelé caméra plénoptique pour de la vision par ordinateur dans des applications robotiques. Plus précisément, pour atteindre cet objectif, nous nous plaçons en amont du côté applicatif, et nous nous concentrons sur sa modélisation pour permettre une estimation de profondeur robuste.Les caméras plénoptiques ou à champ de lumière sont des systèmes d'imagerie passifs capables de capturer les informations spatiales et angulaires d'une scène en une seule exposition. Ces systèmes sont généralement constitués d'une matrice de micro-lentilles (MLA) placée entre un objectif principal et un capteur. Leur conception permet l'estimation de la profondeur à partir d'une seule acquisition. Les contributions clés de ce travail résident dans la réponse à la question "Comment peut-on relier l'information de l'espace monde à celle de l'espace image ?" et surtout, "Comment peut-on relier l'information de l'espace image à celle de l'espace monde ?". Nous abordons la première par le prisme de l'étalonnage, en proposant un nouveau modèle de caméra et une méthodologie pour récupérer les paramètres intrinsèques de ce modèle. Nous exploitons l'information sur le flou de défocalisation là où il était auparavant considéré comme un inconvénient, en le modélisant explicitement. Nous abordons la deuxième problématique comme celle de l'estimation de profondeur, en proposant une méthode métrique d'estimation de profondeur fonctionnant directement avec des images brutes plénoptiques. Elle prend en compte à la fois les indices de correspondance et de défocalisation. Notre modèle se généralise à diverses configurations, y compris la caméra plénoptique multi-focales (en configuration galiléenne et keplérienne), ainsi qu'à la caméra plénoptique monofocale et non focalisée. Avec notre méthode, nous obtenons des estimations de profondeur répétables et exactes (de l'ordre de 1.27% à 4.75% de la distance à l'objet). Elle surpasse les résultats de l'état-de-l'art.Le fait de disposer d'un nouveau modèle complet de caméra et de permettre une estimation métrique robuste de la profondeur à partir d'images brutes uniquement ouvre la voie à de nombreuses nouvelles applications. Il s'agit d'un pas supplémentaire vers l'utilisation concrète de caméras plénoptiques dans les applications de vision par ordinateur

Geometry Preserving Sampling Method Based on Spectral Decomposition for Large-Scale Environments

International audienc

Effect of dehulling and extrusion on nutritional values of faba bean in pigs

Faba bean is a protein-rich feed ingredient for pig nutrition that could replace partially or totally traditional protein sources such as soybean meal. However, faba bean contains secondary metabolites that restrict bioavailability of proteins contained in seed. Technological processes such as dehulling and extrusion may reduce concentration and/or biological activity of antinutritional factors contained in faba bean. Apparent faecal and ileal digestibility of nutrients from whole (W) or dehulled (D) faba bean, without (E0) or with extrusion at two temperatures – 140 °C (E1) or at 160 °C (E2) – were determined in growing pigs. Dehulling increased faecal digestibility of faba bean for dry matter (DM), organic matter (OM) and gross energy (GE). The best amino acid (AA) digestibility coefficients were obtained for diets containing whole faba bean extruded at 140 °C and dehulled faba bean extruded at 160 °C

Blur Aware Calibration of Multi-Focus Plenoptic Camera

International audienceThis paper presents a novel calibration algorithm for Multi-Focus Plenoptic Cameras (MFPCs) using raw images only. The design of such cameras is usually complex and relies on precise placement of optic elements. Several calibration procedures have been proposed to retrieve the camera parameters but relying on simplified models, reconstructed images to extract features, or multiple calibrations when several types of micro-lens are used. Considering blur information, we propose a new Blur Aware Plenop-tic (BAP) feature. It is first exploited in a pre-calibration step that retrieves initial camera parameters, and secondly to express a new cost function for our single optimization process. The effectiveness of our calibration method is validated by quantitative and qualitative experiments

Saccharomyces cerevisiae boulardii CNCM I-1079 supplementation in finishing male pigs helps to cope with heat stress through feeding behaviour and gut microbiota modulation

International audiencePigs subjected to heat stress (HS) decrease their feed intake and growth. The objectives of the experiment were to determine the effects of live yeast (LY) supplementation ( Saccharomyces cerevisiae var boulardii CNCM I-1079) on feeding behaviour, energy metabolism and faecal microbiota composition of finishing boars ( n 10) housed in a respiration chamber at thermoneutrality (7 d at 22°C) or during HS (seven plus six days at 28°C). Dietary LY supplementation increased DM intake ( P = 0·01) whatever the ambient temperature, whereas HS decreased feed intake whatever the dietary supplementation ( P = 0·01). Dietary LY supplementation increased the number of meals ( P = 0·02). Energy retention was higher with dietary LY supplementation ( P < 0·01) but decreased during HS ( P < 0·01). The skin temperature of the supplemented pigs was lower at thermoneutrality and increased during HS to a lesser extent than that of non-supplemented pigs ( P < 0·01). Faecal microbiota composition was determined using 16S rRNA gene sequencing. Treponema, Christensenellaceae R-7 , Ruminococcaceae UCG-002, Rikenellaceae RC9, Clostridium sensu stricto 1 and Romboutsia genera and some bacteria belonging to Alloprevotella , Oxalobacter and Anaeroplasma genera were more abundant under HS. LY supplementation attenuated HS effects on Romboutsia abundance, while decreasing the abundance of some bacteria from Ruminoccocus , Coprococcus , Peptococcus and Oxalobacter genera and increasing the abundance of beneficial bacteria from Lactococcus and Subdoligranulum genera. Our results suggest that higher level of the keystone species Ruminococcus bromii at thermoneutrality may be one of the causes for higher energy retention observed under subsequent HS

L'embouchure du fleuve antique dans les étangs narbonnais

International audienceThe archaeological site of Castelou/Mandirac, located approximately 6 km south of Narbonne, is one of three archaeological research projects focused on the ancient ports of Narbonne. The excavations confirmed the canalization of one of the branches of the River Aude, which was undertaken during the 2nd century BC over a distance of 1,7 km. The river mouth located between the « La Clape » range and the Bages lagoons, plays a key role in the Narbonne harbour complex by maintaining the river’s course and enabling the construction of a large transhipment area.Les fouilles du Castélou/Mandirac, à 6 km au sud de Narbonne, correspondent à l’un des trois chantiers programmés réalisés dans le cadre du Programme Collectif de Recherche sur Les Ports antiques de Narbonne. Ces fouilles ont confirmé la présence d’une chenalisation d’un bras du fleuve Aude dans la lagune, sur 1,7 km. Cette zone d’embouchure, entre le massif de la Clape et les étangs de Bages, joue un rôle déterminant dans le complexe portuaire narbonnais, en permettant de maintenir le fleuve dans son cours et d’aménager un vaste espace de transbordement des marchandises